Add visual highlighting for web math navigation by RyanMcCleary · Pull Request #20372 · nvaccess/nvda

RyanMcCleary · 2026-06-21T23:43:53Z

Link to issue number:

Part of #19191.

Summary of the issue:

When navigating math, there is currently no visual highlighting showing the focused subpart. Visual highlighting is very useful for low-vision users and can make it easier to use speech and magnification together while reading math.

Description of user facing changes:

When browse mode highlighting is enabled, the currently focused subpart is now highlighted while navigating math on the web. Similarly to how highlighting is handled for OCR results, math navigation highlighting is included in browse mode highlighting.

Description of developer facing changes:

The MathCATInteraction class now takes a source object as a constructor parameter (the source object is the root of the MathML being interacted with). A new file source\mathPres\MathCAT\navNodeMapping.py was also added. This file contains functions that are used to map from MathCAT's current navigation node to the corresponding IA2 object.

Description of development approach:

The main tricky part here was mapping MathCAT's current navigation node to the position of that MathML element on the screen. This is done by mapping element IDs to the corresponding IA2 object, from which the position information can be retrieved. One wrinkle in this process was MathCAT creates its own internal IDs for elements that don't already have IDs, so it was necessary to preprocess the MathML to create IDs for elements that don't have them.

Testing strategy:

Unit testing and manual testing in Chrome and Firefox.

Known issues with pull request:

The only issue I know of is the possibility that there might be a more general solution. If it is possible to implement this so that it works for Microsoft Word, Outlook, Acrobat, etc. as well, then that solution might be preferred. However, it seems likely to me that those cases will need to be handled separately.

Code Review Checklist:

Documentation:
- Change log entry
- User Documentation
- Developer / Technical Documentation
- Context sensitive help for GUI changes
Testing:
- Unit tests
- System (end to end) tests
- Manual testing
UX of all users considered:
- Speech
- Braille
- Low Vision
- Different web browsers
- Localization in other languages / culture than English
API is compatible with existing add-ons.
Security precautions taken.

…ghlight-math

CyrilleB79 · 2026-06-22T06:07:33Z

Very nice surprise to see this PR! Thanks @RyanMcCleary.

I have not tested yet but have already somme questions.

Given this PR takes into account a location for the subpart being read in the math equation, what about the new Magnifier? Does it follow? In a foreseable future, it would be nice that magnifier can follow the subpart being read, at least in math exploration/navigation mode.If it follows, to which type of tracking is it tied? System focus (which also includes browse mode cursor when applicable)?

RyanMcCleary · 2026-06-22T12:17:49Z

@CyrilleB79 The magnifier does not yet follow the highlighted subparts. My thinking was that we could handle that in a follow-up PR. As for the type of tracking, system focus seems like the logical choice.

I do still have a couple open questions about this PR where I'd welcome your input. The math highlighter as it's implemented here is a an entirely new highlighter with its own highlighting color (orange). Do you think it should be included in the browse mode highlighter? I can see arguments both for and against this. On the one hand, including it in the browse mode highlighter is simpler and doesn't require adding a new highlighter. On the other hand, math interaction is separate from browse mode, and although it's usually activated from browse mode now, there might be other ways to activate it in the future. Thoughts?

CyrilleB79 · 2026-06-22T13:49:45Z

Yes, Magnifier tracking can be implemented in a follow-up PR. It's even preferable to keep PRs not too big to ease review. I have just mentioned it so that you have it in mind since both features may be related.

My opinion is that browse mode cursor highlighter should be used in this case; the same color should be kept. I agree that this is discutable though.
The reasons why I prefer associating with BM cursor highlighting are:

when navigating math on the web, there is no longer a browse mode cursor and arrows are actually used to navigate in the math expression
we have already another situation where browse mode cursor is used when arrows are being used to move something else than the real cursor: navigation in an OCR recognition result. This would lead to a consistent definition of this BM highlighter highlighting locations where arrows are used and where no system caret is actually moved.

CyrilleB79 · 2026-06-22T13:50:56Z

Note, I've tested this PR and highlighting is working well. Thanks!

RyanMcCleary · 2026-06-22T13:56:17Z

Interesting point re: navigating OCR results. Given that there's already a precedent for this, it does seem like a good idea to use browse mode highlighting. I'll update the PR with that change.

…ghlight-math

Copilot

Pull request overview

This PR adds visual highlighting support for web MathML navigation by integrating MathCAT’s current navigation node with NVDA’s existing browse mode highlighter pipeline, improving low-vision usability while exploring math subexpressions.

Changes:

Add a new vision extension point (post_mathNavigation) and wire it into NVDAHighlighter to drive browse mode highlight rectangles from math navigation.
Enhance MathCAT interaction to preprocess MathML with synthetic IDs and map MathCAT’s current nav node to IA2 object rectangles (web), falling back to the source object rectangle when needed.
Update user documentation and changelog, and add unit tests for MathCAT nav node mapping helpers.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
user_docs/en/userGuide.md	Documents that Visual Highlight (browse mode cursor highlighting) now also highlights the current web math subpart during math navigation.
user_docs/en/changes.md	Adds changelog entry for math navigation highlighting on the web.
tests/unit/test_mathPres/test_mathCatNavNodeMapping.py	Adds unit tests for MathCAT nav node mapping helpers (namespace stripping, synthetic-id removal).
tests/unit/test_mathPres/init.py	Introduces a test package for math presentation unit tests.
source/visionEnhancementProviders/NVDAHighlighter.py	Registers and handles math navigation events, updating the browse mode highlight rect accordingly.
source/vision/visionHandlerExtensionPoints.py	Adds `post_mathNavigation` extension point to notify vision enhancement providers of math navigation rectangle changes.
source/vision/visionHandler.py	Adds `handleMathNavigation` to dispatch math navigation rectangle updates via extension points.
source/NVDAObjects/IAccessible/ia2Web.py	Adds traversal to build a path→(tag, rect) map for IA2 web MathML subtrees.
source/mathPres/mathMlNode.py	Adds shared types for identifying MathML nodes and their rectangles.
source/mathPres/MathCAT/navNodeMapping.py	New helper module for adding synthetic MathML IDs and mapping MathCAT nav node IDs to IA2 rectangles.
source/mathPres/MathCAT/MathCAT.py	Passes a source object into MathCAT interaction, updates highlight on navigation, and cleans synthetic IDs before clipboard export.
source/mathPres/init.py	Extends math interaction APIs to accept an optional `sourceObj` and clears math highlight on exiting interaction.
source/globalCommands.py	Passes a source object into math interaction when available (navigator object or object at review position).
source/browseMode.py	Passes the math object as `sourceObj` when entering math interaction from browse mode.

+	mathElementChildren = tuple(child for child in element if isinstance(child.tag, str))
+	for index, child in enumerate(mathElementChildren):
+		yield from _iterMathMlElements(child, nodePath + (index,))


+def removeSyntheticIdsFromMathMl(mathml: str) -> str:
+	try:
+		root = ElementTree.fromstring(mathml)
+	except ElementTree.ParseError:
+		return mathml
+	for element, _nodePath in _iterMathMlElements(root, ()):
+		if element.get(_NAV_NODE_ID_ADDED_ATTR) != "true":
+			continue
+		originalId = element.attrib.pop(_NAV_NODE_ORIGINAL_ID_ATTR, None)
+		if originalId is not None:
+			element.set("id", originalId)
+		else:
+			element.attrib.pop("id", None)
+		element.attrib.pop(_NAV_NODE_ID_ADDED_ATTR, None)
+	return ElementTree.tostring(root, encoding="unicode")


seanbudd

Note: partial review.

Please review the changes and make sure to:

move out imports if possible, if not leave a comment as to why they are required (E.g. to prevent circular imports)
explain why some code seems to be arbitrarily wrapped in try/excepts with accompanying comments, or remove the try/excepts
make sure to use sphinx style docs rather than epydoc on any changed/added code

seanbudd · 2026-06-23T02:31:52Z

+		sourceObj = self.sourceObj
+		if not sourceObj:
+			return None
+		from NVDAObjects.IAccessible.ia2Web import Math as Ia2WebMath


can these imports be moved out?

The other imports should definitely be moved out, but it seems to me like it would be a good idea to keep this import local. I've added a comment with my reasoning for that. I'm still happy to move it out if you think that's best though.

Co-authored-by: Sean Budd <seanbudd123@gmail.com>

…ghlight-math

…nSourceObj

…ghlight-math

RyanMcCleary and others added 14 commits June 18, 2026 11:41

Wire up math navigation to visual highlighting

10cfd8f

Implement highlight for web IA2

a55dc3a

Debug logging

a0cd4f3

More debug logging

302b494

Store copy of mathML, generating IDs where not already present

1b70876

Fixed math highlight settings

946697a

Change highlight color to orange

c971fe1

Clean up types and move some classmethods into standalone functions

ce5ed03

Refactor nav node mapping

26b22cf

Remove lookup by original ID

83a159a

Remove lookup by original ID

043b31f

Add more test cases

9c51e9c

Merge branch 'highlight-math' of github.com:RyanMcCleary/nvda into hi…

fc92759

…ghlight-math

Pre-commit auto-fix

7a6c02e

RyanMcCleary and others added 5 commits June 22, 2026 10:41

Fold math context into browse mode context

86b691f

Merge branch 'highlight-math' of github.com:RyanMcCleary/nvda into hi…

9f94dad

…ghlight-math

Pre-commit auto-fix

2f8d2b4

Changelog entry and user docs update

f4a1617

Merge branch 'highlight-math' of github.com:RyanMcCleary/nvda into hi…

2401f41

…ghlight-math

CyrilleB79 reviewed Jun 22, 2026

View reviewed changes

Comment thread user_docs/en/userGuide.md

Clean up log messages and docstrings

ae267e9

RyanMcCleary marked this pull request as ready for review June 22, 2026 20:20

RyanMcCleary requested review from a team as code owners June 22, 2026 20:20

RyanMcCleary requested a review from Qchristensen June 22, 2026 20:20

RyanMcCleary requested a review from seanbudd June 22, 2026 20:20

seanbudd added the conceptApproved Similar 'triaged' for issues, PR accepted in theory, implementation needs review. label Jun 23, 2026

seanbudd requested a review from Copilot June 23, 2026 02:19

Copilot started reviewing on behalf of seanbudd June 23, 2026 02:20 View session

Copilot AI reviewed Jun 23, 2026

View reviewed changes

seanbudd reviewed Jun 23, 2026

View reviewed changes

seanbudd marked this pull request as draft June 23, 2026 02:43

RyanMcCleary and others added 16 commits June 22, 2026 23:22

Update source/mathPres/MathCAT/MathCAT.py

4871123

Co-authored-by: Sean Budd <seanbudd123@gmail.com>

Update source/mathPres/__init__.py

25d1d66

Co-authored-by: Sean Budd <seanbudd123@gmail.com>

Update source/mathPres/__init__.py

78684a9

Co-authored-by: Sean Budd <seanbudd123@gmail.com>

Update source/mathPres/__init__.py

b58914f

Co-authored-by: Sean Budd <seanbudd123@gmail.com>

Fix copyright header

5fcd252

Merge branch 'highlight-math' of github.com:RyanMcCleary/nvda into hi…

2f24dd3

…ghlight-math

Merge remote-tracking branch 'upstream/master' into highlight-math

88022c4

Fix vision handler extension point

3a97a79

Move 'import vision' to top level

5ba601d

Remove unnecessary try/except, add justification for local import

61f5629

Cleanup highlight update / clear methods

2bc0545

Cleanup highlight update / clear methods; refactor supportsInteractio…

2543636

…nSourceObj

Merge branch 'highlight-math' of github.com:RyanMcCleary/nvda into hi…

f31bc0d

…ghlight-math

Merge remote-tracking branch 'upstream/master' into highlight-math

19336ad

Merge remote-tracking branch 'upstream/master' into highlight-math

71bacfe

Use a stack for _iterMathMlElements

d11a100

Uh oh!

Uh oh!

Conversation

RyanMcCleary commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Link to issue number:

Summary of the issue:

Description of user facing changes:

Description of developer facing changes:

Description of development approach:

Testing strategy:

Known issues with pull request:

Code Review Checklist:

Uh oh!

CyrilleB79 commented Jun 22, 2026

Uh oh!

RyanMcCleary commented Jun 22, 2026

Uh oh!

CyrilleB79 commented Jun 22, 2026

Uh oh!

CyrilleB79 commented Jun 22, 2026

Uh oh!

RyanMcCleary commented Jun 22, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

seanbudd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

seanbudd Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

RyanMcCleary Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

RyanMcCleary commented Jun 21, 2026 •

edited

Loading